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An information-spectrum approach is applied to solve the multiterminal source coding problem for correlated general sources, 
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I. Introduction 

In this correspondence, we study the classic problem in multiterminal rate-distortion theory, i.e., multiterminal source coding 
problem. In this problem, M (M > 2) correlated general sources have to be compressed separately from each other in a 
lossy fashion, i.e., with respect to a fidelity criterion, and then decoded by the common decoder which has access to a side 
information source that is correlated with the sources to be compressed. This situation is illustrated in Fig. [2 and it is also 
called distributed source coding. The well-known Slepian-Wolf coding problem and the Wyner-Ziv coding problem can be 
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Fig. 1. Separate compression of M correlated general sources with side information at the decoder 

regarded as two special cases of this situation. These two special cases were solved in 1970's for stationary memory less sources 
[1], [2], and later extended to the case of general sources [3], [4]. However, for this general problem, no conclusive results 
are available to date. Even for the special case that the sources are memoryless and stationary and the distortion measure is 
additive, only inner and outer bounds are derived in [5], [6], etc. In this correspondence, we adopt an information-spectrum 
approach to solve this open problem for general sources under maximum distortion criterions. We obtain the rate-distortion 
region for correlated general sources, which is the main contribution of this correspondence. 

The rest of this correspondence is organized as follows. In Section [H] we first briefly introduce required notations and 
definitions in information-spectrum methods [7], and then formally state the multiterminal source coding problem. In Section 
[TT71 the main theorem concerning the rate-distortion region is presented and discussed. All the proofs are finally given in Section 

hyi 

II. Notations and Definitions 
A general source X with alphabet X is characterize by an infinite sequence 

{X n = (x[ n \x«>,:-,XW)}% =1 
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of n-dimensional random variables X n taking values in the n-th Cartesian product X n , and in this correspondence, all 
the alphabets are assumed to be finite. Specifically, for M (M > 2) correlated general sources, each general source X m 
(1 < m < M) with alphabet X m is an infinite sequence denoted by 

fyn _ fyW y( n ) Y( n ) \\°° 

X^-m ~ l A m,li ^-m,2' ' ' ' > A ro,n/Jn=li 

and the whole group of correlated general sources is denoted by (X TO ) me x M > where Xm denotes the set {1, 2, • • ■ , M}. 
Analogously, any part of (X m ) me x M is denoted by (X m ) me A, where A C T M . In most situations, A is also assumed to 
be an ordered set, hence (X m ) me A is virtually a vector. Similar notations apply to any related quantities or functions, e.g., 
(X^) meA and (/("») (X»)) roeA . 

For a sequence of real random variables {Z n }%L lt the limit superior in probability p-limsup„^ oc Z n and limit inferior in 
probability p-liminf„^oc Z n are defined by 



and 



p- limsup Z n = inf < a lim Pr{Z„ > a} = > 

n— >oo I- I"— >°° J 

p- lim inf Z n = sup < a lim Pv{Z n < a} = > 



respectively. Then for general sources X, Y, X and Y, the spectral sup-entropy rate H(X), the spectral conditional sup- 
entropy rate H(X\Y), the spectral sup-mutual information rate I(X;Y), the spectral inf-mutual information rate I_(X\Y) 
and the spectral inf-divergence rate of X conditioned on Y with respect to X conditioned on Y are defined respectively by 

g(X)=p-limsup-ln ) (1) 

H(X\Y) 4 P - limsup - In- ^pr^, (2) 

7(X;V) =p- lim sup -In ^^"l*"' , (3) 

I(X; Y) ± p- lim inf - In X ] Y ^ ' ^ , (4) 

^(X|y|lX|r)^p-liminflln Px " |r "^"! r "; . (5) 

n P^„^„(X"|y™) 

In this correspondence, all the logarithms are calculated to the natural base e. Also note that in these definitions, the general 
source can be replaced by a sequence of random variables with arbitrary alphabets, and we denote such a sequence by 
Z = {Z n } < ^' =1 with alphabets Z = {Z n }%Li, where Z n takes values in Z n . We also call it a general source. 

If a general source is a process, that is, it satisfies the consistency condition, the notation {I n }™j is then replaced by the 
usual notation {X i }°^ 1 for a process and X n = (X\,X2, ■ ■ ■ ,X n ). If the processes (X, Y) are stationary and ergodic, the 
quantities H(X), H{X\Y), I(X; Y) and I(X; Y) are then given by 

H(X) = lim -H(X n ), 

n—*oo n 

H(X\Y) = lim -H(X n \Y n ), 

ri^oo n 

I(X;Y) = I(X;Y) = lim -I(X n ;Y n ), 

n^oc n 



where 



H{xn)=E rp^)V (6) 
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H(X n \Y n ) =E 



In- 



(7) 



P X n\ Y n{X n \Y n ) 

I(X n ;Y n ) = H(X n ) - H(X n \Y n ). (8) 
Moreover, if the stationary ergodic sources (X, Y) are memoryless, we denote them by (X, Y) for convenience, and 

H(X) = H(X), 
H(X\Y) = H(X\Y), 
I(X;Y)=I(X;Y) = I(X;Y) 

due to the memoryless property. 

The multiterminal source coding system can be stated as follows. Given correlated general sources (X m ) me j M and the side 
information source S, the n-length source outputs (AT^) m6 x M are separately encoded into a group of fixed-length codewords 
{4^™\Xm))rn£T M - Then the common decoder observes these codewords (^""'(I^DmeiM and tne side information 5™ to 
reproduce the estimates {Y£) melM = (ijj'f L \s n , ((/>i m) (X™ )) me i M )) m£ i M of X™. Accordingly the reproduced sequences 
form a group of general sources, namely, (Y m ) me j M with reproduction alphabets {y m )m&x M - Here, the encoder ((f>^)mei M 
is defined by 

^:^^X L < m) ^{l,2,...,4 m )} 

and we denote the sequence {</>l"^}^Li and {4> n — {f/)^ )mei M }'n=i by an d 'P respectively. The rate of each encoder 
is defined by 



a ln|^"°TO| 
n 



where \A\ denotes the cardinality of the set A. The decoder (tpn ) m ei M is defined by 
and we denote the sequence 

M m) }^i and {i, n = (v4 m) ) roe i M }^i by and i/> respectively. 

Next, let us define a general distortion measure. A general distortion measure (d^) ke x K is a group of sequences S k ^ = 
{d^j^Li of (measurable) functions defined by 

4 fc) : n n 3c-[o,+c»), 

where if is a positive constant integer and 2^ = {1, 2, • • • , if}. Then the fixed-length coding problem under the maximum 
distortion criterion is formulated as follows. 

A rate-distortion tuple {{R m )mei M , (-Dfc)fcez,c) is .fm-achievable if and only if there exists a sequence (</>, i/>) of fixed-length 
codes such that 

(limsupi?(</4 m) )) me x M < Wmei„, 
(p-limsu P 4 fe )((^) m6 i M ,Vn(^, {^\X^)) meXM ))) kelK < (D k ) kelK . 

n— >-oo 

Here, for any (x m ) melM , (y m ) m ei M G kM > we say that (x m ) melM < (y m ) m ei M if and onl Y if < 2/m for all m 6 2~ M . 
Accordingly, the /m-rate-distortion region and /m-distortion-rate region are defined respectively by 

R-f m {(D k ) ke i K \(X m ) me x M , S) = {{R m ) m ei M \{{Rm) m eiM,( D k)kex K ) is /m-achievable} 
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and 

S) = {(D k ) kl zi K \((R m ) melM , (D k )kex K ) is /m-achievable}. 

The fixed-length coding problem under the average distortion criterion can be defined analogously. A rate-distortion tuple 
((-Rm)mEix ; (Afc)fceijc) i s /a-achievable if and only if there exists a sequence (</>, of fixed-length codes such that 

(UmsupE(^ m ))) me i M 

n— >oo 

(limsu Pj B[4 fe )((^) melM >„(5", (^ m) K))meiM))])tei K < (Afe)*€Z*. 

n — >oo 

It can be easily shown that /m-achievability always implies /a-achievability for bounded distortion measures, and that fa- 
achievability implies /ro-achievability for those bounded distortion measures satisfying J25b defined in Section [TO] when the 
sources are stationary and memoryless. 

III. Main Results 

In this section, we investigate the sufficient and necessary condition on the /m-achievability of rate-distortion tuples, thus 
determining the /m-rate-distortion region and /m-distortion-rate region of multiterminal source coding for correlated general 
sources. The main result is stated in the following theorem and the proof is presented in Section ITvl 

Theorem 1: For correlated general sources (X m ) me x AI , a side information source S and distortion measures (d^)kex K , 
the rate-distortion tuple ({R m )mex M , (Dk)kex K ) is /m-achievable if and only if there exist general sources (Z^ mS> ) me j M 
with alphabet Y[ m e.x M 2< ~ m) = {TlmeiM Z ™ }%=i and a sequence h = {h n }™ =1 of functions h n = {h { ,T' ) ) me x M defined by 



such that 



rn£X M 

(p-lun S npd^((X^) meXM ,h n (S n , (Z^) melM ))) keTK < (D k ) kelK , (10) 



and 



E R m > E I(X m ;Z^) -L W ((Z {m) )meA) ~ L{{Z {m) ) me A;S,{Z^) melM \ A ) (11) 

m£A m£A 

for any nonempty set A C Xm, where 

L^((Z {m) ) m eA)^p-\imM-ln (Z " ^ -— . (12) 

T\ m eAP z ^{Z ( n m) ) 

Remark 1: In Theorem^ we introduce a new quantity l' A '((Z' m ') mei 4), which may be regarded as a generalized version 
of the spectral inf-mutual information rate. Also note that l} A \{Z^ n ^) me A} — when \A\ = 1. 
Remark 2: The condition (|9) may be loosened to the following one 

with lim, woo a„ = liixin^oo /3 n — 1 (see Lemma in Section ITvV 

By Theorem [2 the /?7i-rate-distortion region and /m-distortion-rate region are determined as follows. 
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Corollary 1: The /m-rate-distortion region for a constant distortion tuple (Dk)keic is 

Kf m ((D k )k<Ei K \{X m ) me i M ,S) = [J ^(ztm)) meZM ((A=)fcEZjf), ( 14 ) 

(z(-)) melM 

where 

K (z ^ )m£%u ((D k ) kelK ) ± l(R m ) melM J2 Rm > £ UX m ;Z^)-l^((Z^) meA ) 

- l((Z^) meA ; S, (Z^) melM \ A ), for any nonempty set A C X M X , 

and (.Z( m )) me z M denotes all general sources that satisfy the conditions l|9) and d 1 01 with some sequence h of functions h n . 
The /m-distortion-rate region for a constant rate tuple {R m )mei M is 

T>f 7n ((R m ) me x u \{X 7n ) me i M ,S) = IJ ^(Z(™)) me i M ((^m)mElM). ( 15 ) 



where 



{D k ) k£lK > (pAun S upd£\(XZ) m eiM,K(S n ,(zW) rnelM ))) k 



for some sequence h of functions h n j, 

and (Z( m ^) me x Jf denotes all general sources that satisfy the conditions l|9} and ( II It . 

To have an insight into Theorem ^ we consider some special cases. First consider the case of one terminal with side 
information at the decoder, we then get the rate-distortion function of the Wyner-Ziv problem for general sources. 

Corollary 2 ([4, Theorem 1]): For a general source X, a side information source S and distortion measures {S k ^) ke x K ^ 
the rate-distortion tuple (R, {D k ) ke j K ) is /m-achievable if and only if there exist general sources Z with alphabet {Z n }^L 1 
and a sequence h = {h n }^ =1 of functions defined by 

K : s n x z n -» y n , 

such that 

Px»s»z n = Px»s»Pz n \x», Vn > 1 (16) 

(p-limsnpdi k Hx n ,h n (S n ,Z n ))) kelK <(D k ) kelK , (17) 

#>7(X;Z) -1{Z;S). (18) 

and hence the infimum of the achievable rate for a constant distortion tuple (D k ) kG j K is given by 

mf{7(X;Z)-/(Z;S)}, (19) 

where inf is over all 2 and {h n }^ =1 satisfying properties (I16> and (I17> . 

Second, let us consider the case of two terminals without side information at the decoder. Then Theorem ^ is reduced to 
the following form. 

Corollary 3: For correlated general sources (Xi, X2) and distortion measures k ei K , the rate-distortion tuple (Ri,R 2 , 

(D k ) ke x K ) is /m-achievable if and only if there exist general sources {Z^\Z^) with alphabet {Z^ x Z^}^ =1 and a 
sequence h = {h n }'^L 1 of functions h n — (hn,hn ) defined by 

hn : ZW x Z& -> 3^ x ^ 
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such that 



P Y » Y „ „,(!) 5,(2) - Px?x$ P z m | X „ | X? , (20) 



(p-limsuprfW(^r,X™ I /i„(ZW J ^ 2 )))) feeZK < (D k ) keXK , (21) 

n — >oo 

and 

fl x ^/(X^ 1 ) ) Z( 2 )), (22) 

#2 >T(X 2 ;Z (2) ) -I(Z (1) ;Z (2) ), (23) 

#i+# 2 >J(X 1 ;Z( 1 ))+J(X 2 ;Z^)-7(ZW;Z^). (24) 

If for each fc 6 2^, the distortion measure dn is additive, that is, it is defined by 

1 " 

d^'(x 1 ,x 2 ,y 1 ,y 2 ) = - V dS k \xi,i, x 2<i , yi ti , y 2 ,i), (25) 
n *■ — ' 

i=i 

for all Xi e Af™, x 2 E A^ 1 , t/i 6 and y 2 G D^N where is a nonnegative (measurable) function called memoryless 
distortion measure, then for stationary memoryless sources (Xi, X 2 ), the sufficient condition given by [5] follows from Corollary 

Corollary 4 ([5, Theorem 6.1 ]): For correlated stationary memoryless sources (X\, X 2 ) and memoryless distortion measure 
kei K , the rate-distortion tuple (Ri,R 2 , (D k )k£i K ) is /ro-achievable (or /a-achievable) if there exist a pair of random 
variables (Z\, Z 2 ) with alphabet Z\ x Z 2 and a pair of functions w> x > : Z\ x Z 2 — ► 3^1 and ft/ 2 ) : Z\ x Z 2 — > y 2 such that 

-PxiX 2 ZiZ 2 = PxiX 2 Pz 1 \x 1 Pz 2 \x 2 ' (26) 

(^[d (fc) (X 1 ,X 2 ,/ l ( 1 )(Z 1 ,Z 2 ),/ ) ,( 2 )(Z 1 ,Z 2 ))]) fcelA . < (Dfc)fceij, (27) 

fli>/(Xi;Zi|Z 2 ), fl2>/(X 2 ;Z 2 |Zi), (28) 

Ri+R 2 >I(X 1 ,X 2 ;Z 1 ,Z 2 ). (29) 

The proof is easy and hence omitted here. Note that the memoryless distortion measures are bounded since the alphabets 
are finite, and the sources (Xi, X 2 , Z\, Z 2 ) are jointly stationary and memoryless, which implies the validity of the law of 
large number or the ergodic theorem. 

Though we have determined the whole /m-rate-distortion or /m-distortion-rate region for correlated general sources under 
the maximum distortion criterion, this does not mean that the multiterminal source coding problem has been solved. First, 
the /m-rate-distortion or /m-distortion-rate region in Corollary [2 are obviously incomputable in general. Second, even for 
the correlated memoryless sources, the single letter /m-rate-distortion (or /a-rate-distortion) region is still unknown. It seems 
that the usual treatment for memoryless sources or channels in information-spectrum methods can not be easily applied to this 
problem, and one of the difficulties is how to reduce the general function h n = (hn"^) m £i; M to a memoryless form if we do 
not use the method in [5]. But in any case, Theorem[2does provide a very general sufficient condition, and it also gives some 
hints on the characterization of single letter rate-distortion region. Under the same settings of Corollary 0] we believe that for 
any fm-achievable rate -distortion tuple, there exits a pair of random variables (Z\, Z 2 ) which can be regarded as a mixture 
of infinite general sources satisfying the condition (0, and most of the general sources ( with probability 1 ) achieve the same 
rate -distortion tuple. 
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Finally, let us turn our attention to the multiterminal source coding problem for the mixed sources as an application example 
of Theorem ^ F° r simplicity, we will consider the settings of Corollary [3] and we assume that the source pair is a mixed 
source pair. 

Let (Xi a , X2 a ) and (-X"i/3, -X2/3) be two correlated general sources, and let us define a mixed source pair (X±,X2) by 

Px»x» (xi, x 2 ) = aP x ? a xz a {xi, x 2 ) + fiPx^Xfy {xi, x 2 ) 

for xi e X™, x 2 G X 2 , where a > 0, (3 > are constants such that a + (3 = 1. Then for this mixed source pair, we have the 
following corollary derived from Corollary [3] 

Corollary 5: For given correlated mixed sources (Xi,X 2 ) with component sources (Xi a ,X 2a ) and (Xip,X2p) and 
distortion measures k£T K , the rate-distortion tuple R 2 , (Dk)kei K ) is /m-achievable if and only if there exist 

general sources {Z^\ Z^ 1 ) with alphabet {ZrP x Zn^}^ = i and a sequence h = {hn}^ = i of functions h n = (hn\hffl) 
defined by 

K : z« x 4 2 > -> y? x y$ 



such that 



*x?xzz™z™ - p xrx? p zM\x? P zW\X2> (30) 



(ma^p-limsup^X^,^^ < (D k ) keXlc , (31) 

and 

i?i ^m^T^^W),/^^^))}-/^ 1 );^ 2 )), (32) 

i? 2 > maxf/^; Z( 2 )),7(X 2p ; Z< 2 ')} - /(ZW; Z< 2 )), (33) 

Rx + R 2 > max{7(X lct ; Z^)J(X W ; + nmx{I(X 2a ; Z^)J{X 20 - Z^)} - I(Z^;Z^). (34) 

The proof is similar to the proof of Corollary 1 in [4] and hence omitted here. Lemma 3 and 4 in [4] are applied repeatedly 
in the proof. 

IV. Proofs of Theorems 
To prove Theorem ^ we first need to establish a series of lemmas. 

Lemma 1: Let X = {X n }™ =1 , X = {l n }™ =1 and Y = {Y n }^ =l , Y = {Y„}£Li be arbitrary general sources with 
alphabets X = {X n }^ =1 and y = {y n }%Li respectively, then we have 

D(X\Y \\X\Y) > 0. (35) 

Proof: For any 7 > 0, 

1 V- p , sJ Pxn]Yjx\y) 
> xex n , y &y n l^x n \Y n K x \y) 

< J2 e ~ nip y^v) p x n \ Y Mv) 

xex n ,yey n 
= e-^ Y. P ^V) E P x n \vMv) 

— p—ivy 



1, J 1 1 Px n \Y n {X n \Yn) , x , I ' x <„-.-- .V 

Pr ^- ln n — — f^tt^t < -7 f = >, „ — ! — 7— TT < e 



n Pjt n \Y n (Xn\Y n ) 



hence 
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for any 7 > 0, and this concludes fl35i by the definition □ 

Lemma 2 (An enhanced version of Lemma 1 in [4]): Let U = {U n } n % 1 , V — {V n }%Li and W = {W n } n % 1 be arbitrary 
general sources with alphabets U = {Un}, V = {V„} and W = {W„} respectively, and they satisfy 



Pu„v„w„ > c nPu n v n Pw n \v ri 
with linin^oo c„ = 1. Now let {B n }^ =1 be a sequence of arbitrary (measurable) sets in U n x W n such that 

liminfPr{(C/„,W„) e B n } = e. 
Then, for any 7 > 0, there exits a sequence / = {f n }^Li of functions /„ : V n — > W n such that 



(36) 



(37) 



\fn(V n )\ < 



n(/(V;W)+ 7 ) 



liminfPr{(f/„,/„(K)) G B n } > e. 

71 — >OC 

The proof of Theorem ^ relies heavily on Lemma 1 in [4], but here we choose to present an enhanced version of Lemma 
1 in [4], and its proof is greatly simplified by applying Theorem 5.5.1 in [7]. 
Proof: Let us define a distortion measure 

d n (v,w) = 2J P U„\V n W n (u\v, W)l{(u, w) g B n } 

ueu„ 

for all v G V n , w G W n - Clearly, d n is bounded, and we have 

lim sup E[d n (V n ,W n )} = limsup E Pv n w n (v,w)d n (v,w) = limsupPr{(C/„, W n ) £ B n } = 1- e 

n— >oo ft — >oo ^,ai n— >oc 

v E V n , it? G W n 

by the condition 1371 . Then from the proof of the direct part of Theorem 5.5.1 in [7], it follows that for any 7 > 0, there 
exists a sequence / = {fn}™=i of functions f n : V n —> W n such that 

|/n(V„)| < \e n ^ v -- w ^ 

lim sup E[d n (V n , f(V n ))} < 1 - e. 

n — >oo 

Finally, we have 

Pr{((7„, fn(V n )) G B n } = 1 - Pv{(U n , f n (V n )) £ B n } 

= 1- E ^» E P U n \V n (u\v)l{(uJ n (v))^B n } 



vev n 



ueu n 



(a) 



> 1-C,- 1 E ^V„(«) E ^n|V»W,(«|O./T.(«))l{(«»/«(«))0S„} 



ueu„ 



l-c-^K^,/^))], 



where (a) follows from (I36> . and hence 



□ 



liminf Pr {(f /„, /n(K )) G B n} > 1 - limsup > t _ ^«up gMMl > , 

n *°° n — >oo Cn J-im n — >oo Cn 

The proof is completed. 

Lemma 3: Let (X m ) me x M , & an d )meiM t> e arbitrary general sources satisfying (|9). Now let {B^^Li be a sequence 

of arbitrary (measurable) sets B n in Hmei "^m x S n x llmez M -^n sucn mat 

lim Pr{((X^) melM ,5", (^ m )) melM ) G B„} = 1. 
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Then, for any 71,72 > 0, there exist a group of sequences (/ < - m - ) = {fn 7 ^ }^=i)meiu °f functions /l™ 1 ' : A?" — * Z„^ such 
that 

\fi m) (XZ)\<^ n(I(Xm;Zim))+ll) \ 
lim Pr{((X*) me x M ,S», (/^(X»)) melM ) G S„} = 1, 

n — >oc 

and 

l] A \{f {m) {X m )) meA ) > l} A \{Z^) meA ) - 72 , 

mf (m) (X m )) meA ; S, tf {m) {X m )) melM \ A ) > L((Z^ m) ) meA ; S, {Z^) melM \ A ) - 72 

for any nonempty set A C 1 M , where /< m )(X m ) = {/^ m) (X™)}~ =1 . 
Proof: Supposing that for a given J (0 < J < M), we have 

i/^roi < e r«(7(x m ;z(™))+ 71 )i _ Vm e T/ (38) 

lim PT{((X^) meXM ,S n , (/("*) (X™)) meXjI (^) me i M \ Zj ) G B„} = 1, (39) 

n— »oo 

and 



J'^l (X m )) mGAnIj , (zW) m6X ^) > /l A l((Z(-)) meA ) - ^72, 

I((/ (m) (^™))meAnij, (Z (m) ) me ^\i ; ; S, (/ (m) (X m )) m6 (i M \ A ) n x 7 , (■Z ,(m) )me(x M \A)\Xj) 
> I((Z(™)) meA ; 5, (zWUvi) " ^72 



(40) 



(41) 



for any nonempty set A C Xm. Clearly, the above conditions hold trivially when J = 0, and the lemma corresponds to the 
case J = M. So to prove the lemma, we only need to show that the above conditions also hold for the case of J + 1. 
Now, let U n = ({X^) melM ,S n , (4 m) )mei„\i J+1 ), V n = X] +1 , W n = Z ( n J+1) . From Q, it follows that 



P, 



(x«) melM s»(zi"* ) ) r . 



hence we have 



Next, we define 



e% \2j ( X m )™ex M S n (Z*," 1 ' ) m ei M \i J+1 



Pu n V n W„ = -Pf/„ V„ Pw„ I V„ ■ 



i6Xm\Ij 



G B« 



- {(«,«;) G W„ X W„|tl = ((x ro ) melM , S , (^ (m) ) me i M \x. /+1 ),^ = ((SmW^.S, (^ (m) ) m€ I M \2 J ) G O 



where 



rW = {((x m ) raeIu , s ,(zW) raeIlAIj )e [] z;xs»x [] i# 

^ meijK mei M \i.j 

> 7^l((/M(x m )) me ^ nIj , (zW) m£ ^) _ _L 72; for a n A containing J + l|, 



(42) 
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-In 



) 

-fffnf f < m ) f V" 11 = -r CZ (m) 1 , T >^ ( s > (/n ( a3 m))melj> ) 



S"(/,i" l) (^m))n.ez J (z4 m) ) meXj ,j\i ; ^ ^ " \ x "i))m£2j'\ z )m£l M \Z.j ) j 

( P (/i m) R)) mMnIj (z< m) ) me A\x./^" m) ( a; ™))meAni 7 , (^ (m) )meA\Zj) 

> I((/ (m) (^m))me J 4nIj, (^ <m) )meA\I,; (/ (m) (-X"m ) )m£ (X M \A)flXj > (^ (m) )mG(I M \ A)\Xj ) - ^T 2 ' 



for all nonempty A C Xm j. (43) 

Clearly, by the definition @ and (I12> . we have 

lim Pr{([/„,^„) G O = lim Pr{((X«) m£lM , S n , {Z^) melM \ Xj ) G C;> 

> 1 - lim Pr{((X^) meZM ,^, (/W(X«)) roeXj , (4 m) Wm\x,) £ B «) 

- lim Pr{((z;\)„ ie I, / ^^(z^))„ ieIJlAIJ )^T( 1 )n^( 2 )} 
= 1, 

and hence linin^oo Pr{(L/„, W„) £ C"} = 1. Then from Lemma |2] it follows that there exists a sequence f( J+1 > = 
{fi J+1) }n=i of functions fi J+1) : -> ^ J+1) such that 

c n(7(X J+1 ;Z( J + 1 ))+ 7l ) 



(44) 



Hence, we have 



lim Pr{(pC) m6 z M ,S», / r ( ; 7+1) (^), (4 m) )mei M \x J+1 ) G = 1. (45) 
lim Pr{((X^) me i M ,5",(/( m )(^)) meIj+1 ,(Z( m )) meIflAIj+1 ) G S„} 

n— »oo 

= lim Pr{((X;\)„ leIM ,5^/(■ 7 + 1 )(^) ; (Z^)) meIJAI;+1 ) GC7„} 

> Jirn o Pr{((^)„ leIM ,5^/( J + 1 H^),(4" l) )™ex A Ax ^+1 ) £ O 
= 1. 

Furthermore, Let A be any nonempty subset in Tm- If J + 1 $ A, we have 

£l A l(Cf( TO )(X m )) TOe Anx J+ i, (-^ (m) )™eA\x. 7+1 ) = I |A '((/ (m) (-^m))™eAnXj, (2 (m) ) me A\ij) 

>I |A| ((^ (m) WA)-^72 
>I |A| ((^ M Wa)-^72. 

Otherwise, 

j\A\ ((f(m) (X m )) meAnIj+1 , (Z (m) ) mej4 \x J+1 ) 

1 ? (/f ) R)) mM ni ;+1 (2« m) ) meM i ;H ((/ " m K))™^ni J+1 , (C ) m eA\l J+1 ) 
= p- lim mi — In -. — : -, — r 

n me Anx J+1 ^) ( ^)(/™ m) (^))n meM x J+1 ^)( z » m) ) 

(a) 1 P (fL m) (XZ)UeAni:Azk n> ) m€ A\x ((/» m) ( X m))meAnX J+1 , (^i^ )meA\X J+1 ) 

> p- lim mi — in T7 - — ■ 

— n p z < j+ i>(/1 j+1) (*:; +1 ))^^ 
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1 ^"'(X^Wrvrj^W^^ 

+ p- lim inf — In ■ j—r j—r (J+T) 

n^oo n P (f { ™\x^)) meAnXj (Z { r r ) ) meA \Tj + 1 \zk' + 1) ^" n ( X m))rneAnXj,( Z n n )meA\X J+1 \fn ( X J+l)) 

> i} A K(f (m Hx m )) meAnIj , (Z^) meA \ Xj ) - ^ 72 
+ D((ft\X^)) meAnx AZi^) m ^ Ij+1 \f^ 

>/'^((z(-)W)-^ 72l 

where (a) follows from the property of the limit inferior in probability, (b) follows from (I42> and ( 145 > . and (c) from Lemma 
□ and gfj. 

Analogously, if J + 1 ^ A, we have 

!((/ (-X"ro))meAnXj+i) ^'"'jme^lj+i;^ (/ (m) (^m))m6(l M \A)nIj +1I (2 (m) )mE(I M \A)\Zj + J 

= I((/ (m) (^m))meAnij, (^'"''jmeAyj; 5, (/ (m) (X m )) me( x M \ J 4)ni J+ i I (2 tm) )rae(i M \A)\i, +1 ) 

(a) ..if 

> p- liminf — In [ P, f ( m ) , „„,, r7 (m), ^ifi'ni/v.n , 7 (™k ( 

( z( n n) )meA\I J \S''\ (f$TX X m)) m ^(^M\ J 4-)\^J+l / / 

P (/i m) (^))m 6 Ani J (zi m) ) m6 A\i J ^^" ^^ImeAnlj,^ m) )meA\Ij) 



(Z ( m)) 

P (/^(X»)) m6AnIj (zl™>) m£AX ^ 

(/n m H^m))me(lM\-4)nJj + u (^ ) )me(I M \A)\I W ) 



> 7((/ (m) (X m )) meJ 4nij, (^ (m) )meA\i J ; S, (/ (m) (X jn )) me(ZM \ A)nXj! (Z (m) ) me (z„\A)\Zj) - ^72 
+ iZ((/ (m) (^m))meAnJj, (■Z' ( ' m: ')meA\ij|S', (/ (m) (^m))me(iM\A)nij + i, (■Z' (m '')me(XM\A)\i J+1 || 

(j(™)(X m )) m£AnIj , (Z (m) ) meA \ Tj |S, (/ (m) (^m))me(I„\A)nIj> (^ (m) )me(lM\A)\Ij) 

> I((^ (m) ) m£ A; S, (Z^) me z M vO - ^72, 

where (a) follows from the property of the limit inferior in probability, (b) follows from ( I43> and ( 145 > . and (c) from Lemma 
^and ( I4H . In the same way, it can also be shown that 

(2 (m) )m£(l M \A)\Ij +1 ) 

> I((Z(™)) meA ; S, (ZH)^) - ^±i 72 
for the case of J + 1 e A. 

Therefore, the conditions ( I38> . ( I39> . ( I40i and d41i hold for J+ 1. The lemma is hence proved by simply repeating the above 
argument M times. □ 
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Now, we start to prove Theorem ^ 

Proof of Theorem^} 1) Direct Part: Let 71, 72, 73 and 74 be arbitrary positive real numbers. We define T„ by 



r(m) 

)))fceijf < (-Dfc + 7i)fcex*r 



Clearly, by the condition ( II PL we have 

lim Pr{((X") meZM ,5",(4 m )) meZM ) G T«} = 1. 
Then by Lemma|5J we obtain a group of sequences (/^"^ = {/n m '}™ = i) m ei M of functions fn ■ %m —> Zn"^ such that 

|/^ m) (^")l <e^ 7(Xm;Z<m))+72) l 
lim Pr{((X») meXMI 5", (4 ro) TO WJ G T«} = 1, (46) 

n^oo 

and 

I |A| ((/^(X m )) rneA )>7l j4 l((Z( m )) roeA )-73, (47) 

I((/ (m) (X m )) me A; S, (f (m) (X m )) melM \ A ) > I((Z^) meA ; S, (Z^) meTM \ A ) - 73 (48) 

for any nonempty set A C I M , where /( m >(X m ) = {/^ m) (X^)}- =1 . 

Next, we specify the encoding and decoding procedures. We will present a pair of random encoding and decoding maps 
according to a given rate (R m ) m £i M satisfying 

R m > E ~I(XmiZ^) ~L lA K(Z (m) ) m eA) -I((Z^) meA ;S,(Z^) melM \ A ) (49) 

for any nonempty set A C X M . 

Generation of random bins: For each m G Im, assign each z^ G Z^ to one of the indices in I^tm) according to a 
uniform distribution on T L ( m > independently, where 



e n(fl m + 7l ) 



(50) 



Let g!™'^™)) denote the index to which z^ corresponds. 

Encoding ($1™ : X£ — > l L { m )) me j M . For each to G Xm, the encoder with respect to the m-th terminal is defined 

by 

for all x m G Af™. 

Decoding <&„ : 5 n x U me x M ^ L (^) -> Ilm£X M 3m- The decoder receives the pair (s, ($l m) (a; m )) me i M ) G <S™ x 
rimeiM ^lSt'- For ( s > (*n m) (a3m))mex M )> if there is one and omv one (« (m) )m6i M e Iimez M Z ^ satisfying 

and 

(,,(zW) melM ) er ( 2 ) 
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we declare *„(s, ($! m) (a; m )) me x M ) = h n (s, (z^) me i M ), where 



BCI M ,B^0 



(51) 



T (2,0) 



nlnp /^«)(* (ro) ) 



</(X m ;Z( m )) + 2 72 ,VmeX M |, 



Ti 2 ^^|(,,(^)) melM )e5"x [] $ 
1 



■ In ■ 



P S"(/4" ,) (^)) m£ x f 



n ~ p (/r ) (^ ))meB (^ (ro) )™^)^ (/ ( m ) (x „ ))m£XMXf 



; (s,(^ (m) ) me i M \s) 
> I((/ (m) (X m )) meB ; 5, (/ (m) (X m )) roelMXS ) - 74 |. 

Otherwise, \I/„(s, (<&l m ^(x TO )) me xM) is defined to be an arbitrary fixed element in n TO ei M ^m- 

If the source outputs (x m ) me j M of the m terminals together with the side information s satisfy the following conditions: 

(1) ((x m ) melM ,S, (fi m \x m )) melM ) G n (UmelM X rn X T™) 

(2) There is no (z^) melM G ri me i M ^« m) such that 

( z )meI M ^ (,r> m )) 

(G ( n m \z^)) melM = ($M(x m )) roelM , 

(M^ ra) Wi„)ef, 

then (/n" ) (a;„ l )) me i M is the unique element in l\ meXM Zn" l) such that 

(G ( n m) (fi m Hx m ))) melM = {^\x m )) melM 

and 

(s,(ti m \x m )) meXM )£TW. 

Furthermore, we have 

dW((x m ) melM ,* n (s, (<5>[{ n \x m )) melM )) = dW((x m ) melM ,h n (s, (fi m) (x m )) melM )) <D k + 7l) Vfc G 1^. 
due to ((x m ) melM ,s, 

(fn \x m )) m £2 M ) G Tn \ Hence, every pair ((x m ) ml =2 M , s ) G Ti($ n , * n ) n T 2 ($ n , *„) satisfies 
(^((^Wm, ($£ m) (x m )) ro£ I M )))feeXfir < {Dk + 7i)fe€ijr) 

where 7^($„, \I>„) is defined for the code ($„, *„) by 



3($n,*n) = {((* m ) m€ I M ,s)e LI A '™ X ^ 

melM 



((x m )m£Z M i s ) satisfies condition i 



for i = 1,2. 

Next, we estimate the probability of error. Let us define the probability of error pj™^ (<£>„, \I>„) for the code (<&„, <£„) by 
P e W($„, *„) = Pr{3fc G 1 K , s. t. 4 fe )((^) melM , W„)) > £> fc + 7i}- 
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Then, the P e (n) ($„,*„) can be bounded as follows: 

P e (n) ($„, *») < i - Pr{{{xi) melM ,S n ) e 7i($ n ,* n ) n r 2 ($„, *„)} 

= Pr{((X^) m6lAf , S n ) # Ti($ n , tf n ) n T 2 ($ n , *„)} 

< Pr{(pC) meIju , S") Tx^, *„)} + Pr{((^) m6lM , 5") & T 2 (<f>„, *„)}. (52) 
Clearly, the first term in ( 1521 can be bounded above by 

P r {((X^) melM ,S n , (/^TOWJ T^} + Pr{(5", (/( ro) (X«)) roelM ) T^} 

and hence converges to zero by (I46i and (15 1 1 . As for the second term in (I52> . the event ((X^) me i M , S n ) ^ 72($„, ^n) can 
be rewritten as 

((X™ ) me i M , 5") G |J B), 

where 

B) = {((^U.'l e n x <S"|3(* (m) W G [] 4 m) >s. t. (z (m) ) meB ^ (4 m) (^)) meB) 
(Gi m )( 2 M)) mefl = ($W(, m )) meB] ( Sj (z(-)) mes , (^ m) (x m )) roelMXB ) G 2f>}. 

Then we have 

Pr{((JC) melM ,S n ) ^T 2 ($„,M/„)} < E PrTOmei^flG^,*,^)} (53) 

SCJ M ,B#0 

In order to estimate Pr{((X ? ™ ) m ei M , S n ) £ 72($ rl , ^n)}, let us estimate each term in the summation ( 1531 . For each nonempty 
set B C 2m, we have 

Pr{((X^) melM , 5") G £„(<!>«, * n , B)} 
= E P ^(-9«) E ^-)„ 6lM s-((*mW M ,*)l{3(« (Tn) ) m6fl G I] ZM,S. t. (z^) meB 

* (fk m) (*m)) meB , (g { n m \z^)) meB = {g^\f^{ Xm ))) meB , (a, (^) meB , (4 m) (^)) me z„\ B ) G T^} 
< E Pg -^") E • p (^)„ ielM S"((a;m)meJM,s) 

E l{( 9 W(zW)) meB = (^4 m) (*™))W} 

( Z (m) ) meB 6n meB 2<" i) ,(^ (m, ) me i 3 #(/<" i) ^ m )) meB , 

E P (X») wie x M S«((^m)meZ M ) s ) E 

((* m )»ei M ,«)eII m£% X^xSn (2 (m ») meB Gn meB 2i m > : ( 2 (" l ») meB #(/^)(^ m )) m eB, 

(^( 2 ('™)) meB ,(/<'™)(x m )) meXjvfXB )eT< 2 ) 

Pr{(GM(zM)) roeB - (G ( n m \fi m \x m ))) meB } 
= E B (x „ )m6lM s„((a3 m ) me r M ,s) E ^ 7m 

((^)„ £ I M , S )en m£ l M ^x5» C* (m) )m e Ben m6B ^„ m) ,(« (m) )m S B/(4 m) (a !m ))m 6 S, 1 Ws ^™ 

( S ,(^ (m) ) meB ,(/i m) (^)) me i M \ B )eTf 

^ E P (^) meXj „S"(( a; m)meI JW ,s) E ~ 

(»,^ m, ) meB ,(/i m) (^))™ e i M \ fl )ei'f ) 
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E P (X^) meXMXB S^((x m ) m elM\B,s) ^ — -^y 

(^^"• , )™EB ! (/f ) (^)) me i M \B)eT< !) 

1 



E P ^(/^(x«)) melMNS ( s '( 2;M )-ez„\ B ) E n 

llmeS^™ 



S™(/r"(X™)) mS3 r MNS ^:V~ Tf r (m) 

( s ,( z (™)) melA/ \ B )e5"xn meXjl/ \ B ^ m) (* (m) )meB£n m£ B-4 m) 

( S ,(z (m) Wx M )eT< 2 > 



E 

(a,(*N) m6IjJ ) e r, 



rr t (rn) 

(2) llmeB^™ 



rr r ( " 

.(2,o) nT ,(2,B) llmCS^" 



x s"(/r , (x 5l )) meXjVfX 



E IT r III 

(s ,( z ("-)) melM )eTf' ( "nT( 2 - 



^ 1 " limes V»(x»i( z( ') 



'g"(/r j (x^)) mElAAB K/r j (x^)) m£ B' 



P ?»ffW(y««_, > „iffl"»Vv»M_^ = ( s ' ( 2:(m) )mei A /\s|(2 (m) )mes) 



E 



s r»C7((/("»)(X m )) meB ;S ) (/C"')(X m )) m62:M \ B )-7 i ) pj ^ L 



(m) 



^ . ,1 P M ((z {m) ) cR )>e* ' (m) (*™)WB)-£ meB /(X m ;Z< m >)-2|B| 72 - 74 ) 

< 



i(/((/Cm)(jr m )) m6fl! s,(/(»)(x m )) melM \ fl )- 7 4) n L 

1 ImeS 



(m) 



< e -n(/ |B| ((/ (m) (X m )) meB )+/((/< m '(X m )) me ^^ TJ L (r. 

< e -W£ meB B m -E meB 7(X m ;Z<™>)-/l B l((Z^ 



where (a) follows from the property of the random bins, (b) and (c) from the definition i5l\ . and (d) from J47> . J48t and J50b . 
Since 72, 73 and 74 are arbitrary, let us define 

^ 7i 

72 = 73 = 74 < -7-, 
6 

then 

lim Pr{((X«) 5") G = 

for all nonempty set 5 C 2m, and hence we have 

lim sup Pi") (*„, ¥„) < limsupfpr{((X^) me i M , S n ) £ 2i (#„,*„)} + Pr{((X£) roe x M , 5") T 2 ($„, *„)} 

< limsup Pr{((X") me i M) 5") Ti ($„,*„)} + limsup Pr{((X") me x M) 5") £T 2 ($„,*„)} 

n— >oo n— *oo 

< £ limsup Pr{((X^) melM , S n ) G 

BCZ M ,B^0 

= 0, 

which implies that there exists at least one sequence {(</>„, ipn)}^ = i of codes satisfying 

p-limsupdW((X^) me i M ^n(^ n , (^i m) K)) me iM)) < ^fe +7i, Vfc G J A - 

n — >oo 

and 

lim sup i?(<^ m) ) = R m + 7i , Vm G J M • 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY FOR PEER REVIEW (APRIL 28, 2006) 16 

Finally, let us use the diagonal method to complete the proof. By repeating the above argument with replacing 71 by a 
sequence {7i(i)}^i which satisfies 71(1) > 71 (2) > • • • > and 71 (i) — > as i — > 00, we can conclude that there exists a 
sequence {(<p n , ^n)}£Li of codes which satisfies 

(limsupi?(0i m ))) melM 

n—*oo 

(p- limsup 4 fe )((X™ ) melM , ip n (S n , (</4 m) TO) )))kei K < (Dk)kex K - 

n^oo 

Therefore, the proof of the direct part is established. 

2) Converse Part: Since the rate pair ((i? m )mei M , (Dk)kex K i s achievable, there exits a sequence {((f) n , ip n )}%Li such that 

(limsupi?(^))) me i M 

n— >oo 

(p- lim sup <#> ((X^) melM , ^(5", (^ m) PC)) )))fcez K < (Dk)kex K - 

n^oo 

Let us define the general sources (Z' m ') me i M by Z^™' = 0„(X") (m G Im), then 

holds for all n = 1, 2, • • •. Besides, let us define /i„ = ij) n , then we have 

(p-limsuprfWl^)^,^^",^))^)))^ 
= (p- limsup 4 fc) ((*£W M , ^(S™ (^ m) K)) me iM)))K K 

n^oo 

< (Dk)kei K - 

Furthermore, for any 7 > and sufficiently large n, we have 

E ^ ^ E p (^ ro) ) - \ A \i 

(a) 2 1 

> > p- lim sup — In t-^t 2\Ah 

1 P^,_(^|X«) 

> > p- lim sup — In — - - r—. 2M7 

~ ^ n p . ,(Z {m) ) 

meA Z ( n m) v " I 

= E ~I{X m ;Z^)-2M 1 

> E I{X m ;ZW) - L ]A] ((Z (m) )meA) - L((Z^) m eA; S, (Z^) melM \ A ) 2M 7 

meA 

where (a) follows from [7, Lemma 2.6.2], (b) from the fact that P z (m )|xn (Z^ m) \X r ^) < 1 and \A\ < M, and (c) from the 
nonnegativity of l} A \(Z^) meA ) and L{(Z^) meA - 5, (Z^) meXM \ A ). Since 7 is arbitrary, we have 

E R m > E 7(X m ;Z< m >) -I |A| ((2 (m) ) ra ei) -I((Z( m )) meA ;5,(Z( m )) melMVA ). 

Therefore, the converse part is established. □ 
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